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Abstract 



The purpose of this presehtatibri is ta) to analyze critical ly the 
research and promotional literature on Guilford's Structure of Intellect 
(Sei) model as it relates to cdgnijiiye f uhctioriirig of gifted students, and 
(b) to evaluate the implications of this analysis for the counselor's role 
in the identification and prbgramraihg of gif ted students. This critical 
evaluation is based on a review of proraotibhal literature and available 
research, and is written from developmental and measurement bases. It is 
concluded that the promotional literature promises fC^r mbre than it has 
delivered and that severe psychometric limitations seribusly limits the SOI 
model's usefulness. 
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Identification and Programming for the Gifted: 
Some Limitations of Gaiiford*s Structure of Intellect Model 

The field of gifted education has many unresolved issues. Among the 
most important is the identification and subsequent educational programming 
of gifted pupils. While the concept of int?. 1 1 igence has been broadened over 
the years^ the newer concept remains to be adequately operationalized. 
Educational programs for the gifted vary in their objectives and in the 
instructional components they employ. The variability in educational 
programming is not unexpected in the absence of any cbmmbrily accepted 
operational definition of either intelligence or giftedness. 

One attempt to define the concept of intelligence is reflected in J. P. 
Guilford's (1967) Structure of Intellect (SOI) model. In sharp contrast to 
the concept of unitary or general intelligence (g), Guilford^ in his 
multifactdr theory of intelligence, developed his well-knovm "mental block" 
consisting of three dimensions (operations, contents, and products) and 120 
cells. Consistent with the multidimensional view of giftedness, SOI 
analysts seek individual patterns of relative strengths and weaknesses upon 
which to base an individualized educational program. Theoretically, 
examining cellular structure allows strengths arid weaknesses in cognitive 
functioning to be identified in a highly specific mariner. SOI profiles may 
be developed from several standardized intellectual tests (SOI Learning 
Ability Test, Stanford-Binet, WISG-R, Slosson). Curricular tasks can also be 
arialyzed to identify which particular abilities are stimulated by selecting 
the appropriate curricular tasks. 

The objectives of this paper are twofold: 
a) To analyze critically the research and promotional literature on 
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Guilford's SOI model as it pertains lb the cognitive functioning of gifted 
students; and b) To evaluate the implications of the analysis for the 
counselor's role in the identlficatibn and prdgrararairig of gifted students^ 

Before wo tarn to the review, it Is appropriate to state what specific 
materials we examined. Bastcaiiy the review consisted of three parts. 
First, we read the most recent Product Cataiogae to see what materials were 
published by the SOI Institute. From this catalogue, we reviewed: (a) 
Mary Meeker's textbook (1969) which explores and defines each of the SOI 
abilities^ explains how to teach each of the abilities and describes how to 
make SOI profiles from the Stanf drd-Binet and Wechsler scales; (b) CoiXecied 
Readings ^ Volume I which covers literature on the SOI for the years 1969- 
1975; (c) SOI Research Studies which contains synopses of empirical studies 
from 1981 to 1983; (d) the Technical Data Manual for the SOI-LA Basic Test 
for grades 2 through intermediate; (e) the SOI Form for the Gifted (G) which 
is a subset of the basic SOI-LA test and a group test used for screening 

potentially gifted students; (f) the SOI templates used to derive profiles 

w _ _ 

for the Standford-Binet and the WiSC-R; and (g) the Teacher's Guide for 

Using SOI Test Resul-t^ , which explores the results as they relate to 

l^ fi^ rrilng Styles, learning problems and teaching strategies. 

Secondly, we examined materials that we received from school districts 

that are using or have used SOI materials. These materials include workshop 

materials provided by national figures in the SOI movement as well as other 

training and inservice materials that school personnel received from the SOI 

Institute. 

Lastly, bur review covered the professional literature from journals, 
books, and presentations made at national conventions. While we would not 
want to make the claim that we left no stone unturned in bur search for SOI 
materials, we clearly exerted more effort tb identify arid evaluate SOI 



materials ^han would the typical school district for whom these materials 
are intended. Regretably, there is no single coherent source that provides 
the prospective user with adequate irifbrmatidri upon which to base 
intetitgent decisions regarding use of SOI materials. The evaluation of all 
of these materials was done from both a de ve lb praerita 1 framework and a 
measurement perspective. ^ 

The development framework wilt focus on whether the proposed arid actual 
uses of the SOI model are congruent with what is known about cognitive 
development of children. From a developmental viewpoint, Guilford's theory 
fails to explain how people wind up having 126 different abilities. While 
developmerita 1 ly oriented psychologists, like Piaget, note that not all 
individuals will achieve the highest levels (stages) of thinking, Guilford 
seems to state that intelligerice subsumes all of the 120 abilities naturally. 

the measurement perspective will focus bri the whole notion of the SOI 
model from a psychometric point of view. Hbw adequate are the ribrms for the 
various proposed uses? How reliable are the subtestSi and rabre impbrtaritly, 
how reliable are the difference scores obtained from a series of SOI 
subtests? What is the evidence fnr the SOI model? Does the SOI mbdel give 
us mathematical factors which do not necessarily have 
psychological /educational /instructional meaning? Is there cbmpelling 
evidence for the usefulness of the model? (Even if a difference scores is 
reliable^ it may riot be valid.) What evidence exists that the matching of an 
individual's SOI prbfile to a particular instructional Tnodci enhances 



DEVELOPMENTAL CONSIDERATIONS 



iMPLtCAtlONS FOR INSTRUCTION 

Gtven the availability of SOI educational materials from age 6'months 
onward, advocates of the SOl model seem to convey the impressidri that all 
120 factors are present from a young age even though they do riot attempt to 
assess all "known" ceils at any given age level. Yet, Piaget, a 
developmental psychologist who has given us the most comprehensive picture 
of cognitive developraent, notes that ail abilities are not present in the 
young child. Instead the child develops intelligence over a period of 
several years. Even then, as many as 50 percent of adolescents and adults 
never reach the highest stage of formal operations. No allowance for 
developmental change is made in the SOI model. This approach largely 
ignores the chariges iri irite 1 1 ectua^p^bi 1 ities that occur as the child 



increases in age. How many of the 120 abilities are present in the infant, 

in the preschooler, in the elemeritary school child, in the secondary 

student? No one addresses this issue systematically. Studies by other 

factor analytic theorists indicate that the number of abilities does vary 

with age. For example, research by Thurstone (1938) on the eight •'primary 

abilities" identified in a population of adolescents and yburig adults showed 

that he could isolate only five comparable abilities among five arid 5ix year 

olds. As children grow older, particularly beyond the preado lescent years, 

there is eviderice from factor analytic studies of increasing dif f ereritiatibri 

of intellectual ability (Ausubel and Sullivan, 1970). It is riot at all 
clear from rending the SOI literature, how many abilities have been isolated 

au different ages. Studies using older and hdmbgeriebus subjects, (e.g., 

prbfessibnal3) result in the emergence of numerous specific factors whereas 

studies uPing young and heterogeneous subjects generally prbvide eviderice 

for a "g" factor (Ausubel & Sullivaii, 1976)^ Thus it may be that the general 
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factor becomes differentiated daring the ad6i&>a:cent years. Further research 
Is needed to determine the ex?:ent to which Gaiifo'rd's stractore of intellect 
changes with ^fc^ 



Not drily is-thiare rid systeiratic discussion of developmental change with 
respect to the number df merital abilities (the quantitative aspects), but the 
SOI system does not allow for de ve Idpmerita 1 chariges iri the qua 1 itat i ve 



different at each stage of development. Thought for the infant consists df 
sensorimotor coordinations, is perceptually dominated for the preschooler j 
is logical but concretely bound for the elementary school childi and is 
abstract and pYopos t t iona 1 for approximately half of secondary school 
students. While the SOI model purports to assess different kinds of 
intelligence, this model completely ignores fundamental qualitative 
differences that take place as- the ch il d devel^ops. 

Piagetiari theory is also hierarchi c al * indeed it is difficult to 



imagirie that a child could grasp the nature of conservation without first 
develdpirig ari uriderstaridirig df object permanence, i.ei, an object can exist 
when but of one's visual field. Iri cUr.Lrast td Piaget's hierarchical model, 
the SOI model is cubic (Sternberg^ 1979). 

Closely allied to the issue of chariges iri iritellectual develdpmerit aire 
the issues associated with instructional strategies. Piagetian thedry wduld 
indicate that just as the child's abilities vary with age^ the educatibrial 
implications also vary with developmental status. For instance, prior to 
the junior high period, it is especially important that the child be given 
concrete materials to manipulate in order to proraote a clear and stable 
uriderstaridirig df the world. Children must be given the opportunity to be 
activei td expldre, td tduch, td test, to find ou; and manipulate, because 
the kind of learriirig that Piaget prdpdsed is a prerequisite for later 
verbal, abstract understanding. Td liriderstarid classif icatidri the child must 




aspects of intelligence. Piaget repdrted that thdught is qualitatively 
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have had concrete experience in grbaping objects together in one or more 
dimehsibhs; to arialyzia, she should have palled them apart; to understand the 
ordered properties of number series, she must have Sranged objects in some 
order. Gradually, these actions become internalized and carried out in the 
mind and the child's need for concrete aids diminishes. At this time, the 
adolescent becomes a more abstract learner and greater use can be made of 
verbal presentations. In brief, having recognized a fundamental change from 
concrete to abstract thought, as well as the hierachical nature of thought, 
developmental iy based educators have adjusted their instructional methods 
accordingly. This developmental perspective is lacking in the SOI approach. 
This is no recommendation that methods be adjusted to the child's stage of 
cbgriitive development. One gets the impression that instructional strategies 
are as suitable at one period of development Ss they are at the next or at 
an earlier level, in their stress on int ra-ind i v idua 1 differences, Sei 
proponents have lost sight of noiothetic guidelines. Gifted children pass 
through the Plagetian stages of cognitive develbpmerit at earlier ages than 
those of nor,mal children, but there is no evidence to indicate that the 

sequence As di f f erent_f pr_.Lgi fj:e.d students^ For ^e__. SOI tali^u 1 Jhe_ _ 

instructional methods do not ^cessarily or predictabiy vary with the child 
stage of cognitive develbpmerit. Instead, they vary instruction according to 
the bumps and valleys on the SOI patterriirig. This approacfi represents a 
form of psychometric phrenology except that the diagnosis substitutes the 
feelirig of bumps on the child's SOI profile iristead Of the child's head. 
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THE MYTttXgAI^ CONeEPtlON OF NORMALITY 

Over the years, attempts have been made to use profile analysis of 
intelligence tests such as the Wechsler Scales for diagnostic and 
educational ly 'prescriptive purposes with various groups of exceptional 
children (Clarizib and Veres, 1984). For example, early work on the 
identification of learning disability was based on the implicit assumption 
that children who are having difficulty succeeding in school, which can be 
traced neither to a physical handicap nor to mental retardation, must be 
deficient in some component of cognitive ability. This assuiption, coupled 
with the^'-iiotion that intelligence tests assess the structure of cognitive 
processing, led to the search for characteristic patterns in the 
intelligence test profiles of these children. The widespread use of the 
Wechsler Intelligence Scales (WISG/WISC-R) and the availability of their 
subtest scores led naturally to hypotheses regarding connections between 
particular patterns of scores and learning disabilities. Uneven function 
hypotheses have also appeared in the emotional impairment literature. The 

Tationale~behihd These for^^ clear than in the learning 

disabilities area. To some extent they were guided by the expectation that 
psychosocial deficits would have a more profound effect on verbal than on 
non-verbal abilities, In both areas, the early research results seemed 
promising. Although early work in these areas was somewhat encouraging in 
that patterns that seemed to characterize the subtest scores of learning 
disabled and eraottonaiiy disturbed children were identified, later research 
shoyed these efforts to be misguided (Clarizio arid Veres, 1984; Clarizio and 
Bernard, 1982; Kaufman, 1981.), 

Until recently, it was assumed that the "normal" child had a uriifbrra 

distribution of cbgriitive abilities, kfiei ill, the mean score on each 
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cognitive scale is, In fact^ its ii^rTft. ^^It was riot uritil the mid-1970's that 
the extent of variation in the profiles bf'riorraal children was realized. Up 
to that time, most research proceeded with neither control groups ribr any 
other form of base rate iaformation. It became evident, *.to some, that the 
rather variant subtest patterns that had been thought to be indicative of 
educational disabilities or emotional disturbance 'were present in the 
profiles of many children who were experiencing no educational difficulties. 

In short, variablity as evidenced by peaks (strengths) and valleys 

— _ _ _ _ _ _ _ _ _> _ _ 

(weaknesses) bri profile analysis of cognitive measures is a common attribute 

among nornal and handicapped groups. Moreover, available evidence suggests 

that such variability is everi greater arabrig gifted children (Sattler, 1982). 

Yet, SOI advocates, whb also favbr patterri ahalysiSi seem to regard 

variability particularly below the child's mean in cbgnitive furictibriing as 

a condition in need of special treatment or remediation even thbugh these 

children were selected and tested in the absence of any particular concern 

bn'the part of their teachers, and not because of poor performance in the 

classroom. As diagnosticians, it is understandable bxit—regretable to err by 

attributing pathological significance to various measures of test scatter 

among handicapped children in that the two conditions (irtttatest variability 

and handicaps) cbvaried. It is inexcusable for lis to do so with gifted 

children who are furictibriing well in school. Cbrisisterit with the 

interpretat ibns of valleys bri the profiles bf learning disabled children as 

deviant, the valleys on theprbfiles of gifted children are alsb iriterpreted 

as being unusual if not pathological. This assumption is indeed questionable 

in light of recent evidence that variability in cognitive patterning is far 

more typical than previously believed by psychologists and that this 

variability is even greater among gifted children than among thoae with 

lesser abilities. We must remember that these gifted children were not 

Er|c 10 



identified because of unsatisfactory performance in the classroorai 

r- . ^ . , ...... 

iThe finding of ihtratest variability among gifted youngsters is not 
restricted to traditional individual intelligence tests. Uneven test 
performance also holds with respect to SOI scales. Yet despite the 
awareness by some SOI followers that "for children of very superior IQ 
status unevep performance ts this rule ratii'er chan the exception" on SOI 
measures for both boys and girls (Mtllichamp, no date), the majority of the 
SOI literature suggests that "irreg - lari ties" within each profile "should be 
tlie basis for reraediatidri. " The researcher might be thinking that a little 
extra educattbnal atteritibri, even if it is directed at imaginary strengths 
and weaknesses, certainly can not hurt the gifted student. Therefore, why 
be so critical of profile analysis? Indeed^ what might appear iriridcudus, or 
perhaps even beneficial, is the SOI follower's emphasis bri peaks or 
strengths. It is, of course, appropriate to prbvide ehrichraerit in areas df a 
child's strengths. However, we must first sure that the peaks truly 
represent strengths and are not minor flucuatlbns or chance deviatibhs in a 
child's per f drmance. Even if the strengths' do represent reliable 
"^fTe r e trees ft om tlT^ " c h i"l d ' s b v e FaTT per tormancel tEe chiTd may not 
necessarily be in riecid df enrichment. Statistical differences are not 
necessarily educatibhal ly significant. Given the finite amount of resources 
made available to the gifted in bur society, we can ill afford to 
concentrate our time, energiesi and dbllars bii cbriditidris uhat are not in 
need of special attention. We also run the risk bf making the gifted child 
feel that she is weak or strong in given abilities when this is not the 

case. In short, the unwarranted assumption about the evenness of cbgnitive 

o 

patterning could well result tn a large number of false positives i.e.^ 
children who we say have a changeworthy condition but who in reality are 
ridrraal. 
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PSYCHOMETRIC CONSIDERATIONS 0^ THE SOZ MEASURERS 

For any test to be asefui> it must meet certain basic psychometric 
standards with respect to such factors as reliability,-, validity, and 
.adequacy of normative data. The APA/AERA/NCME Standards £o^ Educational ^ 
Psychological Tests (1974) presents standards for test use as well as for 
test^manuals and is intended to gufide both test developers and test users. 
In general, the SOI-LA does not do as well as most commercially sold tests 
with respect toraeeting the standards stated in that document. We will 
briefly discuss how the SOI-LA fares on four of the topics covered in the 
Standards ; Norms, Reliability, Validity, and Aids to interpretation. 

The Technical Data Manual (1981) reports that the SOI-LA was re-normed 
in 1980. There weiq^ only six sites in the norm group. Three of these were 
located in California. The others were in Indiana, Okiahoraa, and Texas. 
There is ho information presented as to why these sites were chosen, or how 
the children were selected within these sites. There were a total of 2014 
students tested (1015 males^ 999 females) in grades 2 .through 6. The manual 
states that the norms on this SOI-LA test apply to nine of the subtests on 
the Gifted Screening Test although we are riot told how many, if any, in the 
sample were supposedly gifted. No reasbri is giveri as to why they do not 
apply the other three subtests used for form G (NSI, DFU, & DMU). 
Compare this brief description with Standard D2 iri the afore raeritibried book: 
"Norms presented in the tfest manual should refer to defined arid clearly 
described popuiations. These populations should be the groups with whom 
users of the test will ordinarily wish to compare the persons tested. 
Essential" (APA, 1974, p. 20). Further, it is hot at all clear why the 




profile bri form G (copyrighted in 1975) has ndrfis for intertnediate and adult 
levels when the norm sample drily iricliided stiiderits up through grade 6; 
Re ltabiltty ; 

The reliability data presented in the Technical Data Mariual were based 
on a test/retest procedure with a two-td^four week interval. Alterriate form 
correlations are also reported. Reliability data are presented for each of 
26 subtests; for each of the 14 general ability tests, and for the overall 
scale for each grade from grade 2 through grade 6. The manual also reports 
what is incorrectly called the Standard Error of Estimate which was based on 
the test/retest correlations. The more standard statistic to report would 
be the staridard error of measurement, and indeed, a little arithmetic shows 
that Is what is reported — it is just misnamed in the manual! The manual 
reports nirie sets of such statistics for each grade based on nine cells in 
the ndriii groups: Form (Form A, Form B, Total) crossed with sex (Males, 
Females, and Totals). The authors uriderlirie what they refer to as the 
"preferred set(s) of statistics" arid those are tlie dries we Iddked at. The 
sample sizes for the cells ranged from 71 to (fdr both sexes arid bdth 

forms in grade 4). 

In general, the reliabilities (test/retest cor r e 1 a t ioris ) of the 
subtests are quite low. For example, the reliabilities for the DFU subtest 
range from .36 to .60. For the DSR subtest, the test/retest correlations 
ranged from .27 to .55: for CF€ from .15 to .32. Of course, not all were 
this low, but even for the subtest that appeared most reliable (GFU), the 
cdrrelatidris rariged drily frdm .60 to .77. These are hardly high enough to 
alldw much cdrifiderice iri the scdres, arid certainly would lead to unreliable 
difference scores. Alsd, recall that these cdefficients are based on a 
test/retest at a twb-td-fdur week iriterval. Certairily we wduld expect much 
lower correlations for Idriger iritervals. Data drily a few weeks old would 
almost certairily riot be reliable eridiigh fdr decisidri makirig. 




The reliabilities of the general ability composites weire^ as expected^ a 
bit higher. However, even for these composites, the majority of the 
cbrrelatibris appeared to range from the low iSOs to the mid •YOs. Some of 
the correlations ranged into the .20s. The very highest correlation was for 
the Symbolic test (,90 for both sexes for form A in grade 6). But for males 
for Form B in grade 3 the Symbolic reliability was only .5a, 

the reliabilities for the overall scores on the SOI-LA ranged from .63 
to .92. This is lower than one might expect for a test as long as this one 
is. 

The manual does not report the reliabilities of the difference scores. 
However, because the intercorreiations are reported, they can be computed. 
We computed two such reliabilities: one for subtest differences and one for 
composite differences. We used the first Intercorreiation given in the 
matrix for grade 2 in each case. For the difference between DMU and DFU for 
grade 2 we obtained a reliability of .42. For the difference between the 
cbraposite's of raerabry arid cbgriitibri for grade 2, we obtained a reliability of .355. 
(One might think that cbmpbsites shbuld have more reliable difference scores 
than the subtests because they are more reliable. However, this is 
countered, at least in the example abbve^ by higher iritercbrre latibnis 
between the composites which lower the difference score reliabilities.) 

Given the generally low reliabilities of the difference scores arid the 
total number of difference scores an iriditridual teacher might look at giveri 
20 to 30 children each taking all the subtests of the SOI-LA| orie would 
surely expect to see a large number of differences that would most 
reasonably be interpreted as due to chance alone. We find it Interesting 

m 

(and deplorable) that neither the manual nor rhe computer printout of an 
individual's profile mention the problem of unreliable difference scores and 
hbw brie should be cautious when making inferences from observe'd 
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differences—particularly when so many difference scores are being compared. 
It is not even clear how large or small ah observed difference Meeker is 
wit ling to interpret as a true difference. 

Gompartng many different scores on a profile is analogous to running 
muitiple £--tests- Some are almost sure to be "significant" due to chance 
alone. Of course there are ways to control for this (Siiverstein, 1982) but 
this manual does riot even talk about the basic notion of difference score 
reliability let alone considering some of the problems and solutions when 
inferring ah observed difference is, or is riot, a true difference. 

One might compare what was dbrie with the SOI-LA with the Standards 
where it is stated that: "The test manual or research report should preserit 
evidence of reliability, including estimates of the standard error of measuremerit, 
that permits the reader to judge whether scores are sufficiently dependable 
for the intended uses of the test" (APA, 1974, p. 50). If one is to use the 
test for profile interpretation, the reliabilities and the standard errors 
of the difference scores should be reported. 
ReliablLLtjt Tempiata Scares 

In addition to building tests based directly on Guilford's SOI model 
(SOI-LA, SFG), Meeker (1969) has devised a method that uses templates to 
analyze cdmmdrily admiriistered Intel legence tests. Let us consider some of 
the more serious drawbacks associated with SOI profile analyses with the 
Starifbrd-Biriet ^-M, a scale which is said to sample more of Guilford's ceils 
than the WISC-R, the Slbssori, br the Detrbit Test bf Learning Abilities. To 
determine strengths and weaknesses bri the fjrbfiie, Meeker uses a criteribri 
of four months of mental from the child's mental age as d criteribri bf 
significant deviance (Mi 1 1 icharap, no date). To begin with^ a deviation bf 4 
months of mental age represents brie-sixth (16 percent) of the child's total 
mental development for a two-year bid; however, at age 10^ a deviation bf 
four months represents only 3 percent of the child's development. Another 
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complicating factor with the use of a fbiir mbhth criteribh is that this set 
standard does riot coritrol for the variability in IQ distributions that has 
been found to exist at different age levels. Battler's standard deviation 
approach is a sounder method than Meeker's four month criterion. Another 
problem is that psychologists have difficiilty classifying Stanf brd^Blnet 
items in accordance with SOI categories. In one study (Dyer, Neigier , and 
arid Milholland, 1975), nineteen school psychologists attempted to assign 142 
items on the Stanf drd--Binet , L-M, to the five operations categories of 
Guilford's SOI model following charts prepared for this purpose by Meekeri 
On the average, one rater agreed with another on about half of the items and 
their mbdal assignments agreed with Meeker's assignments on only 57 percent 
(N^81) bf the items. These levels bf agriaement aris not considered high 
enough to justify classifying Binet items in accbrdarice with SOI categories. 
Further, many of the items bf the Binet as rescbred have multiple 
classifications for SOI operations. Fbr these items, success br failure must 
be assigned to each of the abilities involved ^ TnTceepirig with the Binet 
procedures. Also, as can be noted from the templates^ the riumber bf 
measures for each SOI ability varies from one age level to another. 
Finally, it should be noted that the Binet measures some SOI abilities more 
thoroughly than others at a given age. To establish the specific effect of 
these inconsistencies on SOI ability scores would require item analysis of 
each child's Biriet profile (Mi 1 1 ichamp, no date). Given the above 
shbr tcbmings , we cannot recommend profile analysis of Binet based on SOI 
categbries. 



The Technical Data Manual says nothing directly about validity. One 



could made some very weak inferences abbut th^^ degree of construct validity 
by cbrrectlrig the Iritercbrrelatibhs fbr attehuatibn but the authors bf the 



Validity; 




i 



manual did hot even do this miich. Wei" did correct the intercorre lat ions 
between cdghitidh and raerabry for attenuation for gradeis 2 and 6. We 
obtained corrected correlations of .68 and .85, Thus^ we cdiild say that the 
data support the SOI model in the trivial sense that the corrected 
correlations are not perfect. On the other hand^ .85 seems quite high. 
Most would suggest correlations as high as .68 or .85 indicate that a 
general factor exists between those two composites. 

Some inferences can be made about the degree of validity of the SOI-LA 
for various purposes by reading the reports in the documents entitled 
CoJ^Lected Readings (SOI Institute, no date) and SQJL Reaearch Studies (SOI 



inferred from these documents. For example the document entitled Co l l ecte d 
Readings-Volume I cbritairies 38 readings. Of these, 23 were authored or 
cbauthbred by Meeker. Only 19 bf these 38 readings were what we would call 
research, and bf these^ bnly fbur were published in a prbfessibrial journal 



The SOI Research Studies document is a cbllectibh bf studies bf the 
sot. These studies are classified into three grbups: Level I — Studies 
focusing on reliability and validity of SOI assessment instruments; Level 
ii — Studies focusing on the efficacy of SOI training-measuring grbwth in 
abilities that results from SGI training; and Level III — Studies focusing on 
the carry-over of SGI training to improved performance in school and work. 

There were eleven Level i studies, eleven Level 11 studies, and seven 
Level III studies. A reasonably typical example of a Level I study is one 
that had as a purpose td determine if certain SOI learning abilities are 
related tb reading achievement. Another Level I study investigated whether 
SOI learning abilities were related tb measures bf self-cbricept. A typical 
Level II study sought tb determine whether SOI trairiiiig prbdtices an Increase 
in learhlhg abilities. -Ah-^xampl^ b a Level lit study was tb determine 
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Institute, rid date). However, it is hard to know just whatv should be 



or book. 
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whether SOI ttairiirig would lead to improvement in arithtnethic achievementi 
The studies described in the dbcumerit are all those collected from 1981 
through 1983. No criteria are stated for the studies beiii^ included in the 
booklet^ except that they are erSpirically based. It is reasonable to assume 
the studies are not an unbiased selection from all relevant studies. Of the 
29 total studies in this document, only 2 appear to have been published in 
professional journals. From reading the short (one or two page) summaries 
of the studies it is usually apparent why they would not have been 
published. Research flaws are reasonably numerous. 

Of the 11 Level 1 studies, none of them actually relate to the 
reliability of profile analysis or the increasing of differential predictive 
validity, although there are some studies shjDWing tHB^SOi-tA predicts school 
achievement, as would any aptitude test. One of the Level I studies shows 
that "various kinds of memory appear tfd be independent of one another." It 
is hot stated how this cdhclusidh was arrived at. Were the correlations 
among the memory tests riot statistically significant? Were they less than 
one? Were corrections made for attenuatidri? Andther Level I study shows 
that SOI-LA subtest scores can be used td "predict teacher perceptidris 
beyond a chance level." this seems a fairly trivial and dbvidus finding. 
Two other Level I studies show a trend iri the directidri the researchers 
expectr.d, but no statistical significance. One other study does not do any 
cross-validation of a regression equation; 

Of the eleven Level II studies, five did not have a control group so 
one could not differentiate such things as practice effects on the test from 
training effects. One of the others had no statistical analysis, one had a 
trend in the 'right' direction, but a nonsignificant statistical test* 
Several of the Level II studies did show that one can raise scores on the 
SOI subareas- by teaching those subareas. This seems much like raising the 
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score on an eye exam by teaching the ietteri; Sn the eye chart. Any time one 
raises the score otl an aptitude Lest by teaching toward the test one should 
consider whether the test has intrinsic or extrinsic predictive validity 
(Gulliksen, 1950). If the test has intrinsic predicti^ie validity, this 
means it predicts because the skills measured bri the test are important 
skills to have in order to do well bri the criteribri measure. If a test has 
extrinsic validity for predicting achievement this mearis that the skills 
measured on the predictor are not in and of themselves importarit fbr success 
on the criterion, but rather, that both measures are related tb same third 
variable. Aithoagh it is a goal worth striving for in test development^ 
most tests do not have intrinsic validity. Training for Improvement on most 
aptitude, tests does no more for achievement than training on an eye chart 
does for vision. 

All of this is recognized at least intaitivtiy by those indivtdaats who 
conduct studies like those categorized as Level ill studies. Do these 
studies iridicate that the s6l-LA tests c.re intrinsically valid? Hone of the 
seven Level III studies were published, although they do seem to provide a 
little suppbrtirig evidence that trairiirig bri the SOI carries over to improved 
performance in school wbrk. Hbwever^ research liraitatidns of the studies 
suggest that the infererices must be made with cbrisiderable caiitidri. For 
example, three of the seven studies had rib cbritrbl group arid brie bf the 
other studies had only a "quasi-coritrbl" group. Orie bf the reraairiirig three 
studies did not show any significant differences between experiraerital and 
control groups on the achievement variable. this leaves us with only two 
studies from which to make inferences regarding the carry-over of SOI 
training the school achievement. in our view, the case for intrinsic 
validity bf the SOi-LA has not been substantiated. 

Research studies in published journals not mentioned in the SOi 
literature are less positive than those that are mentioned. One concurrent 




validity study of the Sei-Form 6 indicates that the muitipie correlation 
between the WISC-R Full Scale IQ and the weighted subtests of the SOi-Form G 
was drily .337, explaining about 11% of the total variance (Stenson, 1982). 
Another study, investigating the efficacy of the SOI as an identification 
tool for the gifted^ indicated that the SOI-LA Form G does not show a strong 
relationship to success in the gifted program. The authors suggested that 
its correlations with academic variables may hot be significant enough to 
warrant its use as an identification todl^ particularly in view of the cost 
per pupil involved in administering the instrument. In regression 
equations, the SOi-tA Form G contributed only minimally to the predictions 
for an annual evaluation of academically gifed students or other academic 
variables (O^Tuel, Ward, & Rawl, 1983). 

A study by Pearce (1983) compared the WISG-R, Raven's Standard 
Progressive Matrices and the SOi-Screening Form for gifted. She found the 
SOI abilities had varying relationships with intellectual functioning as 
measured by the WISC-R arid cdricluded that using the SOI-G for either 
"^screeriirig or ideritif icatiori purposes should be atteraped cautiously. If at 
all. 

Other reviewers of the SOI model arid the SOI-LA test also have been 
sbmewha|^^utions and/or critical. Eysenck (1967; 1973) arid^ Cattell (1971) 
believe that^^the SOI model could be considerably simplified by cbmbiriing 
factors that oV^lap. Horn and Knapp (1973) criticize the subjecti ve 
element in the choiceNand rotation of factors followed by Guilford. This is 
particularly troublesome w^re the factors are rotated to conform with the 
specified SOI modLei._^-M^Nema?:(1964yalso believes the SOl system is far 

too elaborates He objects to ifi^\f racttonization and fragmentation of 

■ . . . . ___ ___ 

ability into more and more factors of and less importance" (1964, p. 

872). ^ 



Vernon (1979) objects to the fact that Gaiiford's approach lacks good 
evidence to prove that the batteries "measure recogni?:ab ly distinct 
abilities in daily life" (p. 60). Vertldri (1965) has suggested that "a large 
propdrtioh of Gdilfbrd's riuirerbus factors of intellect have failed to show 
any external validity which could hot be accounted for by their g^ v^ arid 
space content" (1965, p. 726). Jensen (1980^ p* 225) in discussing the SOI 
model comments as follows: "It has not been satisfactorily demonstrated 
that each of the 120 cells represents a different ability, except in the 
trivial sense, that any tests that correlated with all the other tests less 
than perfectly, after correction for attenuation, represent separate ability 
factorsi The method of factor analysis allows almost infinite subdivisioh 
of abilities if one wishes to identify factors that reliably account for 
almost vanishingly small percentages of variance among all ability tests i" 

Cronbach (1970) also is critical of Guilford's model. "When Guilford 
says that 82 factors of the 120 have been 'discovered' or 'deradnstrated', he 
pro vbkes disagreeraeri t . . . Clearly, Guilfdrd acknowledges a factdr as 
worthy of attention long before other experts are ready to do sd" 
(Cronbach, 1970, p. 339). As Cronbach points out: "The burd_en of proof is 
always cn the person who advocates recbgriitidri of a hew factdr. He ought to 
show that the new factor accoun»:s for an appreciable amount of variance that 
other factors cannot account for, and that the newly proposed factor has 
some interesting correlates with non-test behavior" (p. 342). We^ and the 
reviewers mentioned here, do not feel comfortable that such_^roof_ exi-s^^^ 
Of course ,-^MeeJteii:z:d4-sa^T"eesi S argues in the Tea^cJier '^ Gr ' de , 

"Educators in their search for the best for students are often confused when 
sdme 'expert' in statistics maintains factor analysis (and the SI model) is 
ridt 'valid.' Such statements are not only false, but indicate as well that 
the 'expert' bus ridt fdlldwed SI theory ivor read about SOI practices and 
research" (Heeketi 1981, p. 13). Her defense is weaki The references Meeker 
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cites in the paragraph from which we quoted do not effectively counter the 
criticisms^ Some 'experts' have followed SI theory and SOi practices and 
have remained both doubtful and criticali 

Aids To Iriterpretatibri 

As we have discussed^ the ribrmSi the reliability, and the validity of 
the SOl^LA have all been subject to some criticism. Do these critJcfsms 
have any relevance for sbmedhe ihterpre.tihg the test? Are there statements 
in the SOl-LA materials that relate to how data xrbm the test shbuld be 
interpreted? the APA Standards state that: "The test^ the manual^ the 
record forms, and other accompanying material should help users make 
correct interpretations of the test results and should warn against common 
misuses" (1974, p. 13). Actually, there is little evidence that any 
Information exists that would warn users against common misuses. in fact, 
we would submit that common misuses are indeed encouraged* Recall the quote 
before from Meeker regarding the criticisms of the SOi model- Basically she 
discounts the criticisms and implies that brie wbuld indeed be quite foolish 
to put any stbck in thbse silly statisticians whb misunderstand her woi:k. 

What "aids" are available^ and what db they say? Actually, there are a 
fair number of short papers printed by the SOI institute that presume tb be 
aids to educptdrs. One such 6 page paper is t i t led-J!lJ-sLng^cb^tiirt i ve 
abilities. as_the diagnostl^TasTF'fdr curriculum planning." in this papnr 
Meeker suggests that by using the templates for scoring the Stahf brd-Bihet ^ 
"individual curriculum planning can be done" (Meeker^ hb date^ p. 1.) In a 
section under Convergent Production in that paper she suggests that: 
"Usually the m tperior student reproduces exactly what the teacher wants and 
is often, therefore, misjudged as g±£ted" (p. 3). (italics added). We 
leave it to you to determine why it is a misjudgment to consider a superior 
student giftedi On page 5 of the same document; Meeker states that: "Good 




patterns in divergent prodactton asaaliy are significantly related to 
personality patterns in which good self-concepts exist" (p. 5). This Is an 
interesting statement because in the SOI Research Studies publication, the 
drily study that relates SOI abilities to self concept found that "the total 
self -c brie ept yield bri the Piers -Hariris' scale was found to correlate 
significantly with the SOI bperatibris Membryi Evaluatibri, Cbgriitibri, and 
Convergent Productibri. Little relatibriship was fburid betweeri Total Self*- 
Concept Scores arid Divergent Prbductibri. . Lbw cbrrelatibri betweeri 

Divergent Production and self-concept support the idea that Divergent 
Production. . . is not Influenced by se 1 f -concept" (Study 12/81-6 in the 
SOT Research Studies ) . 

in the publication Using SOI Test Results; A Teacher's Guide (Meeker, 
Copyr ightl979, revised 4/81) many statements are made about what various 
subtest scores mean, and many suggestions are offered to teachers. We 
would consider many ill-founded; Nowhere in this guide are any cautions 
giveri regarding the unreliability of the subtests, or the limited validity 
bf the SOI. Cbrisider the following quote regarding the CSR — Comprehension 
bf abstract relatibris — subtest. "CSR is the mqsjL-uiif^f itral t task on the SOI- 
LA test^ everi thbugh there are brily eight items. [a ribn-sequitdr ] We use 
CSR as an indicator for yburigsters whb say they want tb gb iritb the computer 
sciences, theoretical math and physics. . .. We interpret CSR as an 
ability to discover abstract relationships" (1981^ p. 5). The Technical 
Data Manual shows that the CSR has reliabilities of .26 for both sexes^ both 
forms for grade 2, .OS for males for both forms in grade 3, .28 for both 
sexes for form A in grade 4, and .50 for both sexes both forms for grades 5 
and 6i We submit that most individuals who knew a bit about reliability 
would be somewhat cautious about making inferences about individuals from an 
8 item test with reliabilities this lowi Because most teachers will not 
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read the technicai manual^ it appears very unfortunate, and perhaps even 
unprofessional, not to mention any cautions about interpretations of 'such 
Scores in the Teacher s Gu i de ; 



Finally, we should mention the individual profiles which can be 
obtained as computer printouts. These profiles do aot- contain any band 
iriterpretat iotii there is rid iridicatidn regarding how to irtterptet a 
difference scbre^ arid there is rib statemerit warriirig a user regardirig the 
unreliability of differerice sbcres. 

In cbnclusidni we believe the aids to iriterpretatibri are extremely weak 
with respect to the APA standard qubted earlier that the accbmpanyirig 
material should warn users against common misuses. 

eONGbySiONS and iMPtieATiONS 
Has the intellectual pie been sliced too thin? 6f interest from both a 
theoretical and practical standpoint are the efforts by SQl advocates to 
cbritiriiially reduce the number of factors for which they test. Initially, 
120 factors were identified. Then Meeker identified 26 factors as most 
pritical to schbbl learning and devised the SOI-LA scale to measure those 
"basics" in educatibri. A screeriirig form has ribw beeri devised that tests for 
12 factors. Given this prbgressibri bf everits^ it is cbriceivable that SOI 
researchers will rediscover the "g" factor by the year 2000. It dbes seem 
that Guilford's "mental block" does indeed partition Intelligence into too 
many pieces to have educational relevance. Indeed^ it is not at all clear 
that SGI tests will yield higher correlations than traditional "g" factor 
tests with current or future school achievement. McNemar's (1965) 
conclusion that a large number of Guilford's factors failed to show external 
validity to any greater extent than could be accounted for by the "g" factor 
or by group factors seems as true today as it was when he made^it almost two 
decades ago • 
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/ We offer the following closing cbthmehts. 

li The promotional literature promises more than the use of the SOI model 
itself can deliveri Certainly, the research basts for the promotional 
literature is sketchy i 



2. Sjiecialists in the gifted must take into consideration not only 
quahtitatl v^e^ariat ions but also qualitative variations in assessing 
cognitive develo^^neiit . 



3. SOI advocates assume "^that variablity^ both below arid above the child's 

N. "TV _____ 

mean, is indicative of a deficiency or a strength arid should be remediated 

or enriched (Kester, 1982). Yet, receivt studies have shown that variability 

. _^ 

is more typical than previously believed by psychologists and is even 

greater among gifted students than those of lesser intelligence. 

4. The remediation of deficiencies and the enrichment of strengths is 
predicated upon a basic belief that all gifted children should be well- 
rounded in their cognitive development. in reality, we might well have to 
choose between having a flat SOI profile and a self-actualized child as 
educatibrial objectives. 

5. The SOI model has severe psychometric limitations. These center around 
the lack of reliable assigrimerit of test items to SOI categories^ unreliable 
difference scores, and the lack of adequate validity eviderice for 

differential treatment effects. 

_ _ >^ 

6. More adequate normative data must be gathered in order to Interprf;- 



properly purported strengths an' weaknesses. 

7. The absence of adequately described normative data and data on the 
standard errors of measurement of the profiles constitutes a risky 
practicb when the standard errors pf measurement preclude 
ethical aQd psychbmetrical ly sound Use of the SOI model. 
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8. Much of the limited evidence indicating that SOI abilities can be 
trained and enhance school achievement suffers from a variety of 
methodoiogtcai difficulties. These problems (absence of cbritrdl groups, 
practice effects, regression toward the mean, nonspecific treatment effects^ 
failure to compare SOI teaching approaches with other instructional 
approacltes) render the evidence equivocal. 

9. Given the limited and frequently unsupportive nature of available 
research evidence, SOI analyses are better reserved for research purposes 
than clinical purposes. Cburiseldrs, or other educators, involved in gifted 
programs should hot look upon. the SOI model as one whose practical 
usefulness has been established. 

10. Counselors and other educators interested in the ideritif icatibri and 
education of the gifted must guard against the tendency to use technically 
inadequate tests when more valid instruments are available. Insisting that 
SOI based instruments and analyses meet the minimum standards for 
reliability and validity will go far toward promoting their ethical ani 



suitable usage. 
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